1. Field of the Invention
The present invention relates to a compressed picture editing method for editing compressed picture data, and a picture coding apparatus for mixing a part or an entirety of compressed picture data with other compressed picture data or real time picture data.
2. Description of the Prior Art
A digital video signal possesses an enormous amount of information, and picture coding is indispensable for transmission and recording. Various picture coding techniques have been developed recently, and some of them are produced as encoders and decoders.
As an example, a conventional picture coding apparatus using the MPEG (moving picture coding experts group) method is described below while referring to accompanying drawings.
FIG. 15 is a block diagram of the conventional picture coding apparatus. The apparatus comprises a motion detector 81, a DCT (discrete cosine transform) mode Judging circuit 82, a DCT circuit 83, a quantizing circuit 84, a variable length coding circuit 85, an inverse quantizing circuit 86, an inverse DCT circuit 87, a frame memory 88, and a motion compensation circuit 89. FIG. 16 is an explanatory diagram of the motion compensation predicting method, and FIG. 17 is an explanatory diagram of the frame memory 88 and the motion compensation circuit 89.
Thus in the conventional picture coding apparatus described above, a video signal is scanned by interlacing and is entered as being divided into frame units. A picture in a first frame to be coded, namely, frame t in FIG. 16 is processed by intraframe coding using the data within the frame. First, the DCT mode Judging circuit 82 detects the motion of the input picture data in each two-dimensional block of pixels by, for example, calculating inter-line differences, determines, from the detection result, whether to perform DCT in the frame unit or in the field unit, and outputs the determination result as DCT mode information. The DCT circuit 83 receives the DCT mode information, and performs the DCT in either the frame unit or the field unit to transform the picture data into transform coefficients. The transform coefficients are quantized in the quantizing circuit 84, and are variable-length coded in the variable length coding circuit 85 to obtain a coded bit stream which is sent out to a transmission line. The quantized transform coefficients are simultaneously fed into the inverse quantizing circuit 86 and the inverse DCT circuit 87 to be returned to real time data, and the data is stored in the frame memory 88.
Generally, pictures have a high degree of correlation, and DCT causes a concentration of energy on the transform coefficients corresponding to low frequency components. Therefore, by roughly quantizing the high frequency components which are visually less obvious and finely quantizing the low frequency components which are important components, the picture quality deterioration is kept to a minimum, and the data quantity can be decreased at the same time. In the interlace scanned picture, if the motion is small, the intraframe correlation is strong, and if the motion is large, the interframe correlation is small while the intrafield correlation is high. By making use of such characteristics of interlace scanning, i.e., by changing over the frame-based DCT and the field-based DCT, the interlaced picture can be coded efficiently.
On the other hand, in the pictures after frame t+1, predicted picture values are calculated for every frame, and differences of the predicted picture values from the original picture values, i.e., the predicted errors, are coded.
In MPEG, the calculating method of the predicted picture value includes forward prediction, backward prediction, and bidirectional prediction. FIG. 16 is an explanatory diagram of the prediction methods. The frame at time t is intraframe coded (hereinafter, an intraframe coded frame is called I frame). Then, a difference of the frame at time t+3 from a frame obtained by decoding the I frame is calculated after motion compensation is calculated, and this difference is coded. This operation for predicting a frame which is ahead in time is called forward prediction (hereinafter, a frame coded by forward prediction is called P frame). Frames at time t+1, t+2 are similarly subjected to motion compensation, and difference calculation and difference coding is performed by using frames decoded from the I and P frames. At this time, the predicted picture is composed by selecting in a block having a minimum error from the blocks of I frame (forward prediction), P frame (backward prediction), and the mean of I frame and P frame (bidirectional prediction) (hereinafter, a frame coded at a part thereof or in its entirety by bidirectional prediction is called B frame). The B frame is predicted from frames before and after it in time, so that a newly appearing object can be predicted accurately, thereby enhancing the coding efficiency.
As the encoder, in the first place, the motion vector to be used in prediction is detected by the motion detector 81 on a two-dimensional block by block basis by, for example, the well-known full search method. Next, using the detected motion vector, the frame memory 88 and the motion compensation circuit 89 generates a next predicted value which has been compensated for the motion in each two-dimensional block.
FIG. 17 is an example of the motion compensation circuit 89. Herein the generation of predicted value of bidirectional prediction is explained. The motion vector calculated in the motion detector 81 is fed into an address circuit 882 in the frame memory 88, and pictures of I and P frames stored in the frame memory 881 are read out. At this time, to correspond to the interlaced picture same as in DCT, the two-dimensional blocks are formed in each frame or each field, and the vectors and predicted picture values are generated respectively. In each two-dimensional block, six kinds of errors in total are calculated in square error calculating circuits 893 to 898. The errors calculated by using frame vectors are a forward prediction error, a bidirectional prediction error and a backward prediction error, and the errors calculated by using field vectors are a forward prediction error, a bidirectional prediction error and a backward prediction error.
A smallest one of the six errors is selected by an error comparator 899, and the predicted values and prediction mode information are issued. The above prediction mode information, motion vector, and DCT mode information are variable-length coded in the variable length coding circuit 85, and are sent out to the transmission line together with the DCT transform coefficients.
Thus, according to such a picture coding apparatus, since the predicted error is optimally coded, the energy is decreased and coding of higher efficiency is realized as compared with the case of direct coding of picture data such as intraframe coding. (Refer, for example, ISO/IEC JTC1/SC29/WG11 N0502, "Generic Coding of Moving Pictures and Associated Audio," July 1993.)
However, when editing the compressed picture data coded by using such a picture coding method, various problems occur as a result of the differences of picture data which are coded. FIG. 18 is an explanatory diagram showing a conventional editing method for compressed picture data. Referring now to FIG. 18, the problems are explained below. In FIG. 18, suppose it is desired to link compressed picture data of (a) and (b) at the dotted line portions. Numerals given in FIG. 18 represent frame numbers. Since the B frame is coded after the I and P frames have been coded, the sequence of the coded frames is changed from the sequence to be displayed. After linking the compressed picture data of (a) and (b), the P and B frames right after the editing point, that is, the P frame of frame number 8 and the B frames of frame numbers 6 and 7 in the compressed picture data (b) cannot be decoded because the I frame of frame number 5 used in prediction is lost. Also, the pictures appearing after frame 9 used in prediction of the P frame of frame number 11 cannot be decoded.
Moreover, various problems occur when the compressed picture data coded by the picture coding method is mixed with other picture data. The conventional picture mixing of analog signals is realized by adding elements together, but the compressed picture data are coded in variable length, and cannot be added simply on a bit by bit basis. Besides, in order to perform mixing by decoding the compressed picture to restore real time picture data, adding the decoded picture to other pictures, and coding the added result again, the mixing apparatus must be provided with both a decoder and an encoder, which results in a large size.